Sensitive Discount Optimality via Nested Linear Programs for Ergodic Markov Decision Processes
نویسنده
چکیده
In this paper we discuss the sensitive discount opti-mality for Markov decision processes. The n-discount optimality is a reened selective criterion, that is a generalization of the average optimality and the bias optimality. Our approach is based on the system of nested linear programs. In the last section we provide an algorithm for the computation of the Blackwell optimal policy. The n-discount optimal policies are obtained as by-product of this algorithm. Here we restrict ourselves to the case of completely ergodic Markov decision processes. discount optimality, Blackwell optimality, nested linear programs.
منابع مشابه
Constrained Markov Decision Models with Weighted Discounted Rewards
This paper deals with constrained optimization of Markov Decision Processes. Both objective function and constraints are sums of standard discounted rewards, but each with a diierent discount factor. Such models arise, e.g. in production and in applications involving multiple time scales. We prove that if a feasible policy exists, then there exists an optimal policy which is (i) stationary (non...
متن کاملRisk-Sensitive and Average Optimality in Markov Decision Processes
Abstract. This contribution is devoted to the risk-sensitive optimality criteria in finite state Markov Decision Processes. At first, we rederive necessary and sufficient conditions for average optimality of (classical) risk-neutral unichain models. This approach is then extended to the risk-sensitive case, i.e., when expectation of the stream of one-stage costs (or rewards) generated by a Mark...
متن کاملAverage Cost Markov Decision Processes with Weakly Continuous Transition Probabilities
This paper presents sufficient conditions for the existence of stationary optimal policies for averagecost Markov Decision Processes with Borel state and action sets and with weakly continuous transition probabilities. The one-step cost functions may be unbounded, and action sets may be noncompact. The main contributions of this paper are: (i) general sufficient conditions for the existence of ...
متن کاملRisk-Sensitive and Mean Variance Optimality in Markov Decision Processes
In this note, we compare two approaches for handling risk-variability features arising in discrete-time Markov decision processes: models with exponential utility functions and mean variance optimality models. Computational approaches for finding optimal decision with respect to the optimality criteria mentioned above are presented and analytical results showing connections between the above op...
متن کاملBlackwell Optimality for Controlled Diffusion Processes
In this paper we study m-discount optimality (m ≥ −1) and Blackwell optimality for a general class of controlled (Markov) diffusion processes. To this end, a key step is to express the expected discounted reward function as a Laurent series, and then search certain control policies that lexicographically maximize themth coefficient of this series form = −1, 0, 1, . . . .This approach naturally ...
متن کامل